{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#糖尿病预测-逻辑回归\n",
    "\n",
    "数据说明：\n",
    "Pima Indians Diabetes Data Set（皮马印第安人糖尿病数据集） 根据现有的医疗信息预测5年内皮马印第安人糖尿病发作的概率。   \n",
    "\n",
    "数据集共9个字段: \n",
    "0列为怀孕次数；\n",
    "1列为口服葡萄糖耐量试验中2小时后的血浆葡萄糖浓度；\n",
    "2列为舒张压（单位:mm Hg）\n",
    "3列为三头肌皮褶厚度（单位：mm）\n",
    "4列为餐后血清胰岛素（单位:mm）\n",
    "5列为体重指数（体重（公斤）/ 身高（米）^2）\n",
    "6列为糖尿病家系作用\n",
    "7列为年龄\n",
    "8列为分类变量（0或1）\n",
    "\n",
    "数据链接：https://archive.ics.uci.edu/ml/datasets/Pima+Indians+Diabetes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\xujs\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\fixes.py:313: FutureWarning: numpy not_equal will not check object identity in the future. The comparison did not return the same result as suggested by the identity (`is`)) and will change.\n",
      "  _nan_object_mask = _nan_object_array != _nan_object_array\n"
     ]
    }
   ],
   "source": [
    "# 首先 import 必要的模块\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pregnants</th>\n",
       "      <th>Plasma_glucose_concentration</th>\n",
       "      <th>blood_pressure</th>\n",
       "      <th>Triceps_skin_fold_thickness</th>\n",
       "      <th>serum_insulin</th>\n",
       "      <th>BMI</th>\n",
       "      <th>Diabetes_pedigree_function</th>\n",
       "      <th>Age</th>\n",
       "      <th>Target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.639947</td>\n",
       "      <td>0.866045</td>\n",
       "      <td>-0.031990</td>\n",
       "      <td>0.670643</td>\n",
       "      <td>-0.181541</td>\n",
       "      <td>0.166619</td>\n",
       "      <td>0.468492</td>\n",
       "      <td>1.425995</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.844885</td>\n",
       "      <td>-1.205066</td>\n",
       "      <td>-0.528319</td>\n",
       "      <td>-0.012301</td>\n",
       "      <td>-0.181541</td>\n",
       "      <td>-0.852200</td>\n",
       "      <td>-0.365061</td>\n",
       "      <td>-0.190672</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.233880</td>\n",
       "      <td>2.016662</td>\n",
       "      <td>-0.693761</td>\n",
       "      <td>-0.012301</td>\n",
       "      <td>-0.181541</td>\n",
       "      <td>-1.332500</td>\n",
       "      <td>0.604397</td>\n",
       "      <td>-0.105584</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.844885</td>\n",
       "      <td>-1.073567</td>\n",
       "      <td>-0.528319</td>\n",
       "      <td>-0.695245</td>\n",
       "      <td>-0.540642</td>\n",
       "      <td>-0.633881</td>\n",
       "      <td>-0.920763</td>\n",
       "      <td>-1.041549</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-1.141852</td>\n",
       "      <td>0.504422</td>\n",
       "      <td>-2.679076</td>\n",
       "      <td>0.670643</td>\n",
       "      <td>0.316566</td>\n",
       "      <td>1.549303</td>\n",
       "      <td>5.484909</td>\n",
       "      <td>-0.020496</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pregnants  Plasma_glucose_concentration  blood_pressure  \\\n",
       "0   0.639947                      0.866045       -0.031990   \n",
       "1  -0.844885                     -1.205066       -0.528319   \n",
       "2   1.233880                      2.016662       -0.693761   \n",
       "3  -0.844885                     -1.073567       -0.528319   \n",
       "4  -1.141852                      0.504422       -2.679076   \n",
       "\n",
       "   Triceps_skin_fold_thickness  serum_insulin       BMI  \\\n",
       "0                     0.670643      -0.181541  0.166619   \n",
       "1                    -0.012301      -0.181541 -0.852200   \n",
       "2                    -0.012301      -0.181541 -1.332500   \n",
       "3                    -0.695245      -0.540642 -0.633881   \n",
       "4                     0.670643       0.316566  1.549303   \n",
       "\n",
       "   Diabetes_pedigree_function       Age  Target  \n",
       "0                    0.468492  1.425995       1  \n",
       "1                   -0.365061 -0.190672       0  \n",
       "2                    0.604397 -0.105584       1  \n",
       "3                   -0.920763 -1.041549       0  \n",
       "4                    5.484909 -0.020496       1  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 读取数据\n",
    "# path to where the data lies\n",
    "train = pd.read_csv(\"FE_pima-indians-diabetes.csv\")\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 准备数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "y_train = train['Target']   \n",
    "X_train = train.drop([\"Target\"], axis=1)\n",
    "\n",
    "#保存特征名字以备后用（可视化）\n",
    "feat_names = X_train.columns \n",
    "\n",
    "#sklearn的学习器大多之一稀疏数据输入，模型训练会快很多\n",
    "#查看一个学习器是否支持稀疏数据，可以看fit函数是否支持: X: {array-like, sparse matrix}.\n",
    "#可自行用timeit比较稠密数据和稀疏数据的训练时间\n",
    "from scipy.sparse import csr_matrix\n",
    "X_train = csr_matrix(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 默认参数的Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "lr = LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logloss of each fold is:  [ 0.75974026  0.74025974  0.78571429  0.79738562  0.77124183]\n",
      "cv logloss is: 0.770868347339\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\xujs\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "# 交叉验证用于评估模型性能和进行参数调优（模型选择）\n",
    "#分类任务中交叉验证缺省是采用StratifiedKFold\n",
    "#数据集比较大，采用3折交叉验证\n",
    "from sklearn.model_selection import cross_val_score\n",
    "loss = cross_val_score(lr, X_train, y_train, cv=5, scoring='accuracy')\n",
    "#%timeit loss_sparse = cross_val_score(lr, X_train_sparse, y_train, cv=3, scoring='neg_log_loss')\n",
    "print ('logloss of each fold is: ',loss)\n",
    "print ('cv logloss is:',loss.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logistic Regression + GridSearchCV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "logistic回归的需要调整超参数有：C（正则系数，一般在log域（取log后的值）均匀设置候选参数）和正则函数penalty（L2/L1） \n",
    "目标函数为：J =  C* sum(logloss(f(xi), yi)) + penalty \n",
    "\n",
    "在sklearn框架下，不同学习器的参数调整步骤相同：\n",
    "1. 设置参数搜索范围\n",
    "2. 生成学习器实例（参数设置）\n",
    "3. 生成GridSearchCV的实例（参数设置）\n",
    "4. 调用GridSearchCV的fit方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=3, error_score='raise-deprecating',\n",
       "       estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='warn',\n",
       "          n_jobs=None, penalty='l2', random_state=None, solver='liblinear',\n",
       "          tol=0.0001, verbose=0, warm_start=False),\n",
       "       fit_params=None, iid='warn', n_jobs=4,\n",
       "       param_grid={'C': [0.001, 0.01, 0.1, 1, 10, 100], 'penalty': ['l1', 'l2']},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring='accuracy', verbose=0)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "#需要调优的参数\n",
    "# 请尝试将L1正则和L2正则分开，并配合合适的优化求解算法（slover）\n",
    "#tuned_parameters = {'penalty':['l1','l2'],\n",
    "#                   'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "#                   }\n",
    "penaltys = ['l1','l2']\n",
    "Cs = [0.001,0.01,0.1, 1, 10, 100]\n",
    "tuned_parameters = dict(penalty = penaltys, C = Cs)\n",
    "\n",
    "lr_penalty= LogisticRegression(solver='liblinear')\n",
    "grid= GridSearchCV(lr_penalty, tuned_parameters,cv=3, scoring='accuracy',n_jobs = 4,)\n",
    "grid.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.772135416667\n",
      "{'C': 1, 'penalty': 'l1'}\n"
     ]
    }
   ],
   "source": [
    "# examine the best model\n",
    "print(grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\xujs\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:125: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Users\\xujs\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:125: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAF5CAYAAACiFUGDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3XlclFX7+PHPYRFkERUQdzFzwdxt0dw1Wywzl0rUxKUM\nI+uxtGwxM0u/aj390idts9Qs0tzK8rFcykpxSVLLMHvMJU0REJV9mTm/P+4BBhhWB2aE6/16zWuY\n+z5z5gwWc825z3UdpbVGCCGEEMIRXBw9ACGEEEJUXxKICCGEEMJhJBARQgghhMNIICKEEEIIh5FA\nRAghhBAOI4GIEEIIIRxGAhEhhBBCOIwEIkIIIYRwGAlEhBBCCOEwEogIIYQQwmGcJhBRSkUopU4o\npdKUUnuUUjeV0H60UuqgUipFKfWPUmqZUqpugTb3K6ViLH0eUkrdVbHvQgghhBBl4RSBiFLqQeAN\nYBbQGTgEfKOUCiiifQ9gBfA+0BYYAdwMvGfV5lbgU0ubTsAXwEalVNuKeydCCCGEKAvlDJveKaX2\nAHu11k9aHivgb2CR1nqBjfZPA+Fa65ZWxx4HntFaN7U8/gzw0lrfa9UmCvhFa/1Yhb4hIYQQQpSK\nw2dElFLuQFdge84xbURH24DuRTwtCmiSc6lFKRUE3A98bdWmu6UPa98U06cQQgghKpnDAxEgAHAF\nYgscjwXq23qC1no3MAZYrZTKBM4BicDjVs3ql6VPAKWUl1Kqi1LKq0zvQAghhKjmyvsZ6lZRA6pI\nlnUebwEvA98CDYDXgXeBh6+i607ALiBaKZVc4NwWjBkVIYQQorq7A7izwDEfoAvQA9hd2o6cIRCJ\nB0xAUIHjQcD5Ip4zA9iltf635fFvSqnHgB+VUi9orWMtzy1LnwDBlvsuNs71BuYW81whhBBCGJ+l\n104gorXOUkodAAYAX0LuYtUBwKIinuYFZBY4ZgY0oCyPo2z0MdByvCgnAVatWkVISEjp34S4KlOn\nTuXNN9909DCqFfmdVz75nVc++Z1XrpiYGMaMGQOWz9LScnggYvFvYLklINkHTMUINpYDKKXmAQ21\n1mGW9puA95RS4RiXSxoCb2Jk3uTMeLwFfK+UegpjEWsoxqLYR4oZRzpASEgIXbrYmhQRFcHPz09+\n35VMfueVT37nlU9+5w6TXpbGThGIaK3XWGqGvIJx+eQgcIfWOs7SpD7QxKr9CqWUDxCBsTbkEkbW\nzQyrNlFKqVHAa5bbn8AQrfXvlfCWhBBCCFEKThGIAGitlwBLijg33saxt4G3S+hzHbDOLgMUQggh\nhN05Q/quEEIIIaopCUSEw4WGhjp6CNWO/M4rn/zOK5/8zq8NTlHi3VkopboABw4cOCALnIQQohRO\nnz5NfHy8o4chKkFAQABNmzYt8nx0dDRdu3YF6Kq1ji5tv06zRkQIIcS15fTp04SEhJCamurooYhK\n4OXlRUxMTLHBSHlIICKEEKJc4uPjSU1NldpL1UBOjZD4+HgJRIQQQjgXqb0kroYsVhVCCCGEw0gg\nIoQQQgiHkUBECCGEEA4ja0SEEEJUmMhI4waQng6nTkGzZuDpaRwLDTVuzta3qDwSiAghhKgw1sFA\ndDR07WoED/ZY21qRfQMsX76cCRMmcPLkSbtniog8cmlGCCGEsEEphVIq37H9+/fz2GOPceONN1Kj\nRg1cXV1L1dfs2bNxcXEp8da/f3+7v4+PP/6Yt98udms2h5IZESGEEKKUNm/ezIcffkiHDh1o0aIF\nx44dK9Xzhg8fTsuWLXMfJycnEx4ezrBhwxg2bFju8aCgILuPeeXKlZw9e5aIiAi7920PEogIIYQQ\npfTYY48xY8YMPDw8mDJlSqkDkXbt2tGuXbvcxwkJCYSHh9OhQwdGjRpVUcO9JsilGSGEEKKUAgMD\n8fDwqJTX2rVrFwMHDsTPzw8fHx8GDBjAvn378rW5fPkyjz/+OMHBwXh6ehIUFMSdd97JkSNHAOje\nvTvbt2/n6NGjuZd/2rZtWynjLy2ZERFCCCGczJYtWxgyZAjdu3fnlVdeAeCDDz6gb9++7Nmzhw4d\nOgAwYcIEtmzZwhNPPEGrVq2Ij4/nhx9+4I8//uCGG27glVde4amnniIxMZGFCxeitcbPz8+Rb60Q\nCUSEEEIIJ2I2m5k8eTJ3330369evzz3+yCOP0KZNG1566SU2btwIGAFLREQE8+bNy203ffr03J8H\nDhxI/fr1MZlMhDppLrMEIkIIISpFWppxHxNj/75z+sx5jWvZvn37OHXqFAsWLCAhISH3uNaafv36\n5QYhALVq1SIqKorY2NgKWehaGSQQEUIIUSlOnjTux4yp2Nfo0aPi+q8Mf/75JwAPPvhgoXM5KcUZ\nGRl4eHjw+uuv8/DDD9O4cWNuvPFGBg0axNixY2nWrFllD7vcJBARQghRKYKDjftVqyAkxL59x8QY\nAU7Oa1zLzGYzSikWLVpESBG/qBo1agAwevRo+vXrx4YNG9i6dSvz589n/vz5bNq0iX79+lXmsMtN\nAhEhhBCVomZN4z4kxH7VT4t6jWtZixYtcheVlqbAWcOGDYmIiCAiIoLY2Fg6duzIvHnzcgORgkXZ\nnI2k7wohhBBOpFu3bjRp0oQFCxaQZmPRS3x8PADZ2dkkJyfnOxcUFERQUBAZGRm5x7y9vbl06VLF\nDvoqyIyIqHyyU5UQ4hp1+vRpPv74YwB+/vlnAF577TUAmjVrxhg7LIBxc3Pj/fffZ8iQIbRv356x\nY8fSsGFDzpw5w7Zt22jUqBGrV68mISGBVq1acf/999O+fXu8vLzYsmULv/32G0uWLMntr2vXrnz5\n5ZfMmDGDTp064efnx1133XXV47QXCURE5avonaqEcAK7p0Ti+rkRcJuzzJiSUnH19cLF3ZiINt0f\nyq2LJeC+1pw4cYKZM2fmu9zx0ksvAdCnT58yBSK29rLJcfvtt7N7927mzJnD4sWLSUlJoUGDBnTv\n3p3w8HAA/Pz8mDRpElu3bmXt2rVorWnZsiUffPAB48ePz+3rySef5MiRI7z33ntcvnyZ1q1bSyAi\nhBBV3a2LQ8ESaMR8Ek3ImK7ELDpAyGgJuK8VYWFhhIWF5TvWp08fzGbzVfft7++PyWQqtk3nzp3z\n1REpyNPTk4ULF5b4Wr6+vkTmzEI7IVkjIoQQQgiHkRkRIaoDWZcjHKTgf3qtWsGMGfb5T68i+xal\nY30JMiYtsVx9SCAiRHUg63KEg1RkMCCBhuNZX4I88+onMPOnMvfhNJdmlFIRSqkTSqk0pdQepdRN\nxbT9SCllVkqZLPc5t18LtPuXUuqoUipVKXVaKfVvpVTlbJsohBBCiBI5RSCilHoQeAOYBXQGDgHf\nKKUCinjKE0B9oIHlvjFwEVhj1ecoYJ6lzzbABOAB4LWKeRdCCCGEKCunCESAqcC7WuuVWuujQDiQ\nihE8FKK1TtJaX8i5ATcDtYHlVs26Az9prVdrrU9rrbcBn1naCiGEEMIJODwQUUq5A12B7TnHtNYa\n2IYRTJTGBGCb1vpvq2O7ga45l3iUUtcBg4Cv7TFuIYQQQlw9Z1isGgC4ArEFjscCrUt6slKqAXAX\nMNL6uNY60nJp5ydlVIxxBd7RWs+3y6iFEEIIcdWcIRC5WuOAROAL64NKqb7A8xiXefYB1wOLlFLn\ntNavFtfh1KlT8fPzy3csNDSUUFmeLa5RU96N5POjRopddkYGbqE3kf3Bc7h5GGu3728TyuJH5b9v\nIUTpREZGFiqSdu7YmXL15QyBSDxgAoIKHA8Czpfi+eOBlVrr7ALHXwE+1lp/ZHl8RCnlA7wLFBuI\nvPnmm3SRtEZRhSx+NJTFGIHG5nc+YdDkMWxeuopB4aMdPDIhxLXI1pfzda9+woiZZd9rx+GBiNY6\nSyl1ABgAfAlguZQyAFhU3HMtsx4tgGU2TnsBBYMTc07/lnUoQgghhCinKT9v4PP4iwCkeP5Zrj4c\nHohY/BtYbglI9mFk0XhhyYJRSs0DGmqtwwo8byKwV2sdY6PPTcBUpdQhYC/QEmOW5MsSg5DHHoN6\n9YyfpWKOqAIiY2OJvHABgDON/Zm6YgXedf1551ej9E5ovXqEBhWclBTi6kX+Gknkb8YUfnp2Oqcu\nn6KZXzM83Yzyp6HtQgltX76/sRXZN8Dy5cuZMGECJ0+epGnTpuXux1FGjhzJoUOHiImx9RFpH4tv\nHMpiy8/T9/wfr5ejD6cIRLTWaywLS1/BuCRzELhDax1naVIfaGL9HKVULWAoRk0RW+ZgzIDMARoB\ncRgzLi+WOKApU2C0TFmLqiM0KCg30Fj0cwxPNm/KWycu8MQ9dzp4ZFWXdfCX4J9C/IoVBPin4F/N\ngr/Q9nnBQPS5aLq+15XI4ZF0aXD1l78rsm+wvTvu/v37+eijj9i3bx+HDx/GZDKVuHkdwOzZs5k9\ne3aJ7fr27cuOHTvKPWZrSilcXByeHFsipwhEALTWS4AlRZwbb+PYFcCnmP5ygpA59hqjEEKU2oUd\nYPm2nnomjmM1XfCKNeOfGGicbxcKQTLbeq3ZvHkzH374IR06dKBFixYcO3asVM8bPnw4LVu2zH2c\nnJxMeHg4w4YNY9iwYbnHg+wYnK5atYprYRWC0wQiQghRlVh/W1/00RqebF6P8Scu8EToAw4embga\njz32GDNmzMDDw4MpU6aUOhBp164d7dq1y32ckJBAeHg4HTp0YNSoUaXqIy0tjZo1a5Z6rK6urqVu\n60gSiAghRDWitcYMmLQmW+vC92D7uI1zMVeuOPjdVL7AwMBKeZ1u3bphNptZvHgxTz/9NNHR0fzr\nX/9i7ty5rFu3jmXLlnHw4EEuXrxI06ZNmThxIs8880y+S0kF14j88ccfhISE8J///AeAN954g3/+\n+YfOnTuzdOlSOnbsWOK4euz+CtcLv9k8l1E7rVzvVQIRIYQoA6016WYzKSYTqWYzqSYTKTn3RRz7\noY43ABvq+vDXn3+W+IGfc748wULB87bO2U0pZwNE2SmlOH/+PIMHD+ahhx5i3LhxNGrUCIAPP/yQ\nOnXqMH36dLy8vNi6dSvPPfccqamp+dah2FrjArBs2TLS09OJiIjAZDIxf/58RowYwbFjx2y2t1ZP\nZeHpkmnz3IWMJC6V471KICKEqFKyyhgklPVYqtlMaa66uwBerq54u7iQ5W1kccTUrEH8pUu4Aq5K\n4aZU4XvIfVzDxSXfeetzJT234H1pnqu1iczsdDKyU8nISiU9K5X0rBTSslJIz0omLSuF1KxkUjOT\nSc28wrnLJ4iqoH9HAWfPnmXFihWMGZO/Nsf69evx8MjbSP7RRx9l/PjxLFq0iFmzZpW4QPX8+fMc\nO3YMb28jQA4ODmbkyJF899139O/fv9jnbug+tMg6W9N3X8NZM0KI6sGsde6Heb4PfDsGDtmlXJzn\n6eKCt4sLXq6ueLm44O3qmhs4eLm6UtfNrdAxW+2KOubh4pL77TJnjcjzZy/yxO232fV3qrUmJSuF\nKxlXuJJxhaSMpNyfC96SMos+dyXjChmmjCJfR6Go5VGLWh618PXwpZZHLSi6uU1p2cbUfUy8/dNJ\nc/rMeY2qwNfXl9E2Mjitg5Dk5GQyMjLo2bMnK1eu5Pjx4/kWxdoyevTo3CAEoFevXmit+euvv0oM\nRCqCBCJCCMD4QMswm4sMEuxxLN1sLtVYXAFvV1fjA97GB32AuzvNriJI8HJ1xaWEKeiKlmXKshkY\nFBlIZNo+n5SZhFkX/Xv1cPXIDSCsb41qNSKkRojNczmBhvXN29270LR9dHQ0XWd2LfV7PnnpJABj\n1pe9+mZZXqNHkx4V1n9latKkic1LJYcPH+bFF19k586dJCUl5R5XSnH58uVS9WutTp06ACQmJl7l\niMtHAhEhqrhMs5nfU1L4JTmZny6eZ03dS5CSxvQgd57dugGTiztm5Y5JuYMqXc0BF3MmrjoLF52F\nq/XPOgsXcxauOhNXnZ173k9nUbfAeeNcltVzMy3nsnDB9gerCUiy3K4lf8fGwsVs5qQnMff1J7iS\ncaXYb+4Kha+HL741CgcEjXwbGcGCjXMFgwnfGr54uHkU+TqVLbh2MACrhq0iJCDErn3HxMcwZv2Y\n3NeoCmxlyCQkJNC7d2+CgoKYN28ewcHBeHp6EhUVxUsvvYS5FMF+Udk0jkr1lUBEVDrrQk/pKSmc\nWrGCZikpeFazQk8V4Up2NgeTkzmYnMwvlvsjKSlkaY0CPLPiycpKALMntTPTCKhVEzdtwk1n4Uo2\nbtq45ftZZ+NGlnGvs3HFRLnnEnL2wc7H1XLzLP8bd3JJF7LAty4dFPS78ea8YKGIYMK7hjcupQwK\nryU13YwP1pCAELsVHSvqNaqqbdu2kZSUxPbt2+naNW826siRIw4c1dWRQERUOusqn9H799O1aVMi\nPT3p0r69g0d27dBacy4zMzfY+CUpiYPJyRxPTweghlK09/bmRl9fHm7QgM4+Pmw5uIRXd79ARL2Z\n/CekOy+cuMATD0pNi4qSE3BnZUGK6QLNk+JIuL4O++o0BIyAe7gE3KKMcmYzrGc+MjIyeOeddxwy\nnnx7zfiUJ2dGAhEhnJ5Ja/6XlpYbbPxiucVlZQFQ282NTj4+3BsQQCcfHzr7+NDGywt3q5XzUX9H\nMXfnTJ7v9Tz1Tt3gqLdSpcXFwe+/Q0yMcf/770HExATxzz/52/3hCQEBcCYQlgdCoNUtICD/48BA\nqF0broEq3dXG6dOn+fjjjwH4+eefAXjttdcAaNasWaEMF3vr3bs3vr6+hIaGMmXKFLKzs1m5cmW+\nBayVyXqvmXU/f8KIcvQhgYgQTiTdZOI3y3qOnNmOw8nJpFi+/TTx8KCzjw+TGzaks48PnXx8aObp\nWWzu/+X0y4xaP4qbGt3ErD6zWLpyQ2W9nSpHa/jnn4IBh/FzfLzRxtUVWraEkBAYPx7atoXzB7fQ\nYeHrfDvhZRp37ElcnBG4xMfD339DdLTxOCHBeA1rrq7g7184QLEVtAQGGm3d3Sv/d1NdnDhxgpkz\nZ+b7f+6ll14CoE+fPmUKRIqq82F9vqB69erx1VdfMW3aNF544QXq1q3L+PHjueWWWxg8eHCJfRT1\neiWNpSJJICKEg1zMyiq0niMmJQUTRg2KNl5edPbxYVhAAJ19feno7U1AjRpleg2tNZO/nszFtIvs\nGLsDd1f5hCoNsxlOnSocbPz+O+QUE61RA9q0MQKO224zAo6QECMIKfjPtPlKArexncybxjMovGeR\nr2sywcWLRoCSE6xYBy05Px87lvezZWIsn9q1bQcpRQUvZagaXq2EhYURFpZ/0/c+ffqUakFoSfz9\n/YvdLC8qqugKLT179mTPnj2FjhfsLzIyMt/j1q1b23xNDw+PUm3cV1EkEBGigmmt+TsjI996jl+S\nkzmdYRRhqOniQgdvb3r6+fF4o0Z09vGhvbc3Ne2wT8TKQyuJ/C2SyOGRNK/T/Kr7q2qys+H48cIB\nR0wMpFmSWry9jQAjJASGDMkLOJo3Bzc7/wV1dc0LDkJKkVSiNSQlFQ5aCgYuhw7l/ZySUrgfL6+y\nBS61aoGDs59FFSKBiBB2lG0280eB9RwHk5O5mJ0NgL+bG519fXmwXr3cSyutvLxwrYC/6n8m/EnE\n5gjGdRrHyHYjjYNmM36x52njkkadfxKMT9xqIDMTTp40gg7r26lTkGmZUajlCy1awKAWMOU24+fr\nWkCD+jbWaGQDf5b+9b3PnQXAMzHBiAY8PIxbjRpX9YmulBEU1KpljLc00tJKnnE5fhz27DF+tlVa\nwt3dCFKsamIVKfLXSCItuxCnZ6fTyr8VM7bNwNPNyJIKbZe3OWBZVWTfovJIICJEOaWYTBwucGnl\n15SU3KJdzT096ezjw78aN84NOhp5eFTKddhMUyah60Jp4tWAtwPHweuvww8/wI8/EnbpErmTzS8+\nXeFjcQY1gFaWW5GSgIOWm531sdz3f/5JeP7JAoOrkReU5AQoV3srpq+aHh408fCgSU0PaOkB7Qq0\ncXfPFxxlZxtrV2wFLr//Dv/7X/Hv3XoXYnuryL5F5ZFARIhSiLNOlbXc/5GaisbYv6OtZT3HqHr1\nctdz1HbEisGMDPj5Z7Z/MIPXformtnM1cU3paywCuPVWeOopNlzOYGFwff51LpEH7h5Q+WO0g6Qk\nOHkKTp2EEyeM2Y6TJ+Hc+bw29QKNyyfNmhn3wcHGzc+v8sd78pujBL8ykdPTF9O0RxPj38nWLTOz\n6HMZGZCaakxRFNcm53Y16xisAhO3GjUI8vAgyEZQE52ezhr7/ZpENSWBiBBWtNacSE8vtJ7jn0xj\nt0kfV1c6entzW506TG/ShE4+Ptzg7Y2Ho/IrU1Nh717YudOY8YiKgvR0etSA+C5tcH14HPTuDV27\n5q6g/PujNUQ1r8dI7wtGcOLEclJirReLxsSQmxKrlBFkhIRA214wuG3eeo5atRw7dmtpJ4xLBSkd\nb4UhFVPIqxCTqXQBS2luRQVIsbGV816E0/hl8IukevjbPBefeN7m8ZJIICKqrUyzmZjU1NxgIyeD\n5Ypl9Xj9GjXo7ONDWP36uZdWWtSs6dg9SpKSYPfuvMBj3z4jbaJuXejVi6SXn2P4ubdw7dyFr8d+\nk1uy3bqa7ZnAWrQ6fZrlgfXY5gTVbK1TYgumxSYkGG1yUmLbts1LiW3bFlq1MhZaChtcXY1fTkX+\ngqKjjSBXVBteV2LxcU21ec4nveR9bmyRQERUC1eyszlko/R5pqVoQ8uaNens48OMpk1zg476DioQ\nlE9iIvz0U17gER1tfNMNCjJmOt5807i/4Qa0UoyMvIeDLq4cHv5xvhLh1tVsN+86zKDJYWxeuopB\n99xZaW8lJyXWVg2OnJRYDw9o3doIMnJSYtu2heuvL5wSK4SofK13vk+nLrZn9Y6/+gnMLHtBNwlE\nRJVzzjpV1nL/P0suZg2laOftTRcfHyY2aEAnHx86eHvja+88zPK6cAF+/DEv8Dh82JgyaNwY+vSB\nRx4xAo9WrQplWyza8xab/9zM16O+pr5PfQe9gbyU2IIBx9GjhVNi27aF++7LCziaNze+yAshqg8n\n+esrRNmZc0qfF9hvJdZS4cnP1ZXOvr7c4++fO8sRUqD0ucOdPWsEHDmBR046bYsWRsAxdapxHxxc\nbJrnwfMHeWbbMzx5y5MMajmo0HnrNMeLVy7wTihcvLKYdyJXA+VLc8zIMAprFQw4jh3LK7JVu7YR\nYHTpAmPG5AUcjRtL2XIhhEECEXFNSDeZOFJgPcchq9LnjS2lzydZlT4PLqH0eaXT2kjtsA48jh83\nzoWEGAHHzJnQq5fxSV1KKZkphK4LJSQghPm3zbfZxjrNMeaTaEKe7UrM3UsICS154WRKijGbUTDg\nOH48LzGjXj0jwOjdG8LD84p+1a8vha+EEMWTQEQ4nURbpc9TU8nWGhegtSVV9r6AgNygo6ylzyuF\n1sb0gHXg8fffxidz+/YwaJDxyd27t/FJXk5Tv5nKqUunODDpAB5u5V/XculS/syUnIDj1Km8No0b\nG0HGoEF5sxshIcb+JkIIUR4SiAiHumD5Sv1eZiaxv/3GweRkTlq2sve0lD6/tVYtIho1opOl9LmX\nsy4iMJvhyJH8gUdsrHENoksXeOABI+jo2dPIcrGDtb+v5f3o93nvnvcICSxFTXDgUpIrO+nNjm0B\nJO7NCzjOnTPO56TEtm1rDDkn4GjTxrlSYsU1IjLSuAGkpxuRbbNm4GmkNBMaatycrW9RaSQQEZUu\nJiWFdXFxrI2L45Bl9eJnWVncZDIxIjAwr/R5zZq4OfNCApMJDh7MCzx+/NHYsczdHW66CSZMMAKP\nHj3A19fuL3/68mke2fQIw0OG83CXh0v1nC++gBGPdyCbnbh9rLnekhI7cWLe4tHWrWUTNGFH1sFA\nTrpvZKQRnDtz38Dy5cuZMGECJ0+epGnTpnbpUxQmgYiocFprDlsFHzGpqfi4ujLY35/Q7GxmZGSw\n3cuLrh07OnqoxcvKgp9/zgs8du0y8k49PaFbN5gyxQg8unWr8OIWJrOJMevH4FvDl/cHv1+qtTAH\nD8KoUdC382Xe+vlWspZ9QsewzhU6TiGuZUqpfP9vaa1ZsWIFGzZs4JdffuHixYs0b96ckSNHMm3a\nNDyKSfmfPXs2s2fPLvE1+/bty44dO+wy/hwff/wxV65cISIiwq792osEIqJCaK35OSkpN/g4np5O\nbTc3hvj7M/+66xhYpw6erq5E79/PjIwM51pUmiM93ahamhN4REUZlUy9vY1ZjmefNQKPm24yCmBU\notd+fI1df+/i+7DvqVOzTontz5+HwYONWY/5k0/QdmIMMW66EkYqRNWRmprKhAkT6N69O5MnT6Ze\nvXpERUUxa9YsduzYwfbt24t87vDhw2nZsmXu4+TkZMLDwxk2bBjDhg3LPR5UAYUFV65cydmzZyUQ\nEVWfWWuirlxhXVwc6+LiOJ2RQYC7O0MDAng7MJB+tWtTw5kvtaSkGFVLcwKPvXuN0tZ+fkYmy+zZ\nRuDRpYv9938vg12ndzF752xe7PUivZr1KrF9Wpqxfb3ZbFyaufK9BCBClEeNGjXYvXs33bp1yz02\nceJEmjVrxssvv8yOHTvo37+/zee2a9eOdu3a5T5OSEggPDycDh06MGrUqAofuzNzmk8FpVSEUuqE\nUipNKbVHKXVTMW0/UkqZlVImy33O7dcC7fyUUm8rpf5RSqUrpY4qpSqvlGQ1kG02811iIhHHjtE4\nKoqev/zCZxcuMNjfnx0dO3Kue3fea92aO+rWdb4g5PJl+PprY2ajWzej6MXtt8M770BgICxcCL/8\nYtQZ37QJpk2Dm292aBByKf0So9aPonvj7szsM7PE9lobS1V+/RW+/BIaNaqEQQpRRbm7u+cLQnIM\nHToUrTUxOXWA7GTXrl0MHDgQPz8/fHx8GDBgAPv27cvX5vLlyzz++OMEBwfj6elJUFAQd955J0eO\nHAGge/fubN++naNHj+Li4oKLiwtt27a16zivllPMiCilHgTeACYB+4CpwDdKqVZa63gbT3kCeNbq\nsRtwGPKHNjB/AAAgAElEQVQ2glRKuQPbgPPAMOAfoBlwqSLeQ3WSZTaz49Il1sbFsTE+nvisLJp6\neDCyXj2GBwbSvVYtx+7HUpT4+PxVSw8eND6pGzQwqpaOG2fMeISEOGXxC601j371KJfTL/PDuB9w\ncyn5f985c+Czz+Dzz2VLECEqyjlLyllAQIDd+tyyZQtDhgyhe/fuvPLKKwB88MEH9O3blz179tCh\nQwcAJkyYwJYtW3jiiSdo1aoV8fHx/PDDD/zxxx/ccMMNvPLKKzz11FMkJiaycOFCtNb4OWIL6mI4\nRSCCEXi8q7VeCaCUCgfuBiYACwo21lonAUk5j5VS9wG1geVWzSZajnXTWpssx05XxOCrg3STia2J\niayNi+PLhAQuZWdzfc2aTKxfn+GBgdzo6+t86zzOncufSmv5hkBwsBFwPP64cd+ihVMGHgV9dPAj\n1hxZw5oRa2hWu1mJ7desgVmzjGBkxIhKGKAQJcmp8W/nmYN8fea8RiVasGABfn5+3HXXXXbpz2w2\nM3nyZO6++27Wr1+fe/yRRx6hTZs2vPTSS2zcuBEwApaIiAjmzZuX22769Om5Pw8cOJD69etjMpkI\nddJUZocHIpaZi67A3JxjWmutlNoGdC9lNxOAbVrrv62ODQaigCVKqSFAHPApMF9rbS6us8fS0qjn\nBLuSOlqKycSWixdZGxfHVwkJJJtMtPXyYkqjRgwPDKSDt7dzBR+nThkBR07w8eefxvFWrYyAY8YM\n4/4aTMM7Gn+UKf+dwsTOE7n/hvtLbL9/P4SFGVkyL7xQCQMUojROnjTux5R9Y7QyvUaPHhXXfwFz\n585lx44dLF26lFp2KrSzb98+Tp06xYIFC0jI2YIaY1a0X79+uUEIQK1atYiKiiI2NrZCFrpWBocH\nIkAA4ArEFjgeC7Qu6clKqQbAXcDIAqeuA/oDqyznrweWYrznOcX1OcXDg9Ht25dm7FXOlexsvk5I\nYG1cHP+9eJE0s5lOPj4826QJwwMDCfH2dvQQDVrD//6Xf8YjpwRou3YwcCC8+qqxyLRBA8eO9Spl\nZGcQui6UJrWa8Nadb5XY/swZY3Fqp06wbNk1MdkjqovgYON+1SrjEqg9xcQYAU7Oa1SC1atXM3Pm\nTB5++GEmTZpkt37/tHyJevDBBwudy0kpzsjIwMPDg9dff52HH36Yxo0bc+ONNzJo0CDGjh1Ls2Yl\nz5o6C2cIRK7WOCAR+KLAcReMYGaS1loDvyilGgPTKCEQeePNN1n9+ef5joWGhjrttNbVSszK4ktL\n8PHtxYtkas3Nvr68HBzM8MBAWjhDdSutjfKf1oHHuXNG1dJOnWDoUGOdR8+eYMfrtM7gue3P8Xvc\n7+yZuAfvGsUHgikpcO+9xnrajRvzCkwK4RRy/paEhNit6FiRr1HBtm7dSlhYGIMHD2bp0qV27dts\nNqOUYtGiRYQUEbDVsGxrMXr0aPr168eGDRvYunUr8+fPZ/78+WzatIl+/frZdVzWIiMjicypamtx\n7tiZcvXlDIFIPGACCs4pBWEsNC3JeGCl1jq7wPFzQKYlCMkRA9RXSrnZaJ/r6alTGf3QQ6V46WtX\nXGYmG+PjWRsXx45LlzBpTQ8/P+Zfdx3DAgNp6uhPMJMJDh/OX7U0Pt74hL3xRnjoISPw6NHDSK+t\norb8bwtv7nmTf9/+bzo3KL74mNkMY8ca29vs3g3X6CytEE5v7969DBs2jJtvvpnVq1fjYueMwBYt\nWuQuKi0qHdhaw4YNiYiIICIigtjYWDp27Mi8efNyA5GKuIRu68v5ulc/YcTMsl92c3ggorXOUkod\nAAYAXwIo47c2AFhU3HOVUn2BFsAyG6d3AQWnMFoD54oLQqqyfzIyWB8Xx7r4eH64ZCQP9a1dm7eu\nv56hAQE0qOSiXPlkZ+cvHvbTT0Z6bY0aRmpteLgReHTrBj4+jhtnJYpNjiVsYxh3Xn8nT3Z7ssT2\nM2fChg3GTIhlQb1wIOttUDIvXEcNviBz8XXUWG0ck21Qrk0xMTHcc889XHfddWzatKnYaqrl1a1b\nN5o0acKCBQsYPnw4NQvM8sTHxxMQEEB2djbp6en4WP1NDAoKIigoiIyMjNxj3t7eXLrkvAmjDg9E\nLP4NLLcEJDnpu15YsmCUUvOAhlrrsALPmwjs1VrbWoK9FIhQSi0CFgOtgOeA/1ch78BJnUpPzy0w\ntvvKFdyUYkDt2rzbqhVDAgIIdOSutX/8QdBHH/HN/v10PHLEWO1esybceis8/bSxsPSWW6rl9QWz\nNhO2MQyFYvmQ5bio4r9xrVoFc+fCggXGpRnhePkCjei/oOsQWHKg4i5JiAqXnJzMHXfcwaVLl3jm\nmWf46quv8p1v0aKFzTojZeXm5sb777/PkCFDaN++PWPHjqVhw4acOXOGbdu20ahRI1avXk1CQgKt\nWrXi/vvvp3379nh5ebFlyxZ+++03lixZkttf165d+fLLL5kxYwadOnWya4aPPThFIKK1XqOUCgBe\nwbgkcxC4Q2sdZ2lSH2hi/RylVC1gKEZNEVt9nlFK3QG8CRwCzlp+LpQOXNX8mZpqBB/x8fyclISH\nUtxRty4r2rRhsL8/ddzdHTe4s2dh9Wr49FM4cID6Xl4cat+e8xMm0Gj0aKPYhSODIyfx//b8P745\n/g1bRm8hyKf4ayy7dxub1o0fb9RcE0JUjISEBM6ePQvAjBkzCp0PCwsrUyBScC8ba7fffju7d+9m\nzpw5LF68mJSUFBo0aED37t0JDw8HwM/Pj0mTJrF161bWrl2L1pqWLVvywQcfMH78+Ny+nnzySY4c\nOcJ7773H5cuXad26tQQitmitlwBLijg33saxK0Cxc/Ra673ArXYZoJP7PSWFtZaZj8MpKXi5uDDI\n35+nGzfmbn9/fB1YDZRLl2DdOiP4+O47Y3fau++GGTM4HBTE3SYTB7y9aXRTkcV0q5Xoc9HM2DaD\np7o9xR3X31Fs21On4L77jImjpUslQ0YIewoLCyMsLG8ivlmzZphMpmKeUXr+/v4l9tW5c+d8dUQK\n8vT0ZOHChSW+lq+vb6GFpc7EaQIRUTZaaw4lJxvBR3w8R1NT8bXsaDsrOJg769bFy9XVcQNMSzPK\np3/6qXGflQX9+sEHH8CwYUY5dUDv32+keggAkjOTCV0XSrt67Zg7YG6xbZOS4J57jCUz69dX+r57\nQghhFxKIXEO01uxPSsqd+fgrPZ06bm4MCQhg4XXXcZtlR1uHyc42Zjw++cT4ZExKMjJc/u//4MEH\noWFDx43tGvHkf5/kzJUzRE+KxsOt6MjCZDKKlZ0+bWwKXMUylkVVYr1qNz3dKDA4Y0be2q+rWbVb\nkX2LSiOBiJMza83uy5dZGxfH+vh4/s7IINCyo+1wy4627o7cTE5r2LfPmPlYvRpiY+H66+Gpp4w/\nAK1LrEknLNYcWcOHBz9k2b3LaB1Q/O/t2Wdh82ZjssnJ9q8SIr+KDAYk0KgSJBBxQtlmMzsvX2Zd\nXBwb4uM5n5lJgxo1GB4YyPCAAHrVro2roxcDHD1qBB+ffgrHj0P9+sYfhFGjjFmQYsYXGRtL5IUL\nAKSnp9Pq9GlmNG+OZzUuq3/y0kkmbZrEAzc8wPhOhZZE5bNsGbzxBrz1FtxZ2r2krb45NrqQzh+0\notHiGbBavjkKIRxLAhEnkWk2s8OyqdzG+HgSsrNp6uHBKMuOtt2cYUfbs2eNrVw//RSio6FWLRg+\nHN59F/r2hVJeFgoNCsoLNKKjjU1RDhyAalpWP9uczej1o6ntWZt373m32OJDO3caJVUefRSmTCnD\ni1gFGv+LNpKTDiyRTFIhhONJIOJAaSYT3yYmsi4uji/j47lsMnF9zZo80rAhwwMC6OoMO9omJuZl\nvHz/vZFae889xk5qgwZVyxof9jZn5xz2nNnDD+N+oLZn7SLbHT9urPPt3RsWL5YMGSFE1SCBSCVL\nMZnYnJDAurg4vr54MXdH2ycbN2Z4YCDtnWFH27Q0+OorY9Hp5s3Gysj+/Y1rAkOH5ma8iKv346kf\nefXHV5nVZxY9mha9Y+ilS0b85+8Pn39uZEALJycLKYUoFQlEKsHl7Gy+sgQfW6x2tJ3RtCnDAwJo\n4ww72mZnw44dRvCxYYOR8XLTTUapzgcfvOZ3sHVGiWmJjF4/mp5Ne/JCrxeKbJedbfwTnD9vVMGv\nW7cSBynKTwINIUpFApEKcjEriy/i41kXF8fWxMTcHW1nBwczzJl2tN27Ny/j5cIF41vb008bi05b\ntnT0CKssrTWPbHqE5MxkVg1dhatL0etrpk6F7dvhm2+Mfx4hhHAWu6dE4vq5MfOXlJZYrj4kELGj\nWMuOtuvi4tiRmIgZ6OHnx4IWLRgWEEATZ1lPEROTl/Hy11/GbMeYMUbw0aWLLD6oBO9Hv8+6mHWs\ne2AdTfyaFNluyRL4z3/gnXdgwIBKHKAQQpTCrYtDYbEx8+cebVkJX0YSiFylszk72sbF8ePlyyig\nT+3aLG7ZkvscvaOttTNnjIyXTz6BgwfBzw9GjID33zd2tXVkIbRq5ve43/nXln8xqcskhoUMK7Ld\n1q3wxBPG7dFHK3GAQthRvnR9s5lT6ek08/TE01L/6GrS9Suyb1F5JBAph5NpaayzzHxEXbmCu1IM\nqFOH91q3Zoi/PwHOsmnbxYtGxssnn8APPxgZL4MHw0svGRkvzhIkVSPp2emErgsluHYwb975ZpHt\njh6F+++HgQONmiFCXKus0/Wjk5LoeuAAkW3b0sXX16n7Bli+fDkTJkzg5MmTNG3a1C59VqaRI0dy\n6NAhYmJsbVDvPCQQKaVjOTvaxsVxIDkZD6W4s25dVlp2tK3tLGkMqamwaZNx2eW//zUyXgYMgA8/\nNDJe/PwcPcJq7dmtz3I0/ij7Ht6Hl7uXzTYJCUa82KiRMYnlyP0KhajOCu6Oq7VmxYoVbNiwgV9+\n+YWLFy/SvHlzRo4cybRp0/Ao5svd7NmzmT17domv2bdvX3bs2GG38bs4svJ2KcmfuCJorfk9NTV3\nX5dfLTva3u3vz/SmTRlUt65jd7S1lp0N27YZwceGDZCcDDffDK+/Dg88YFQ9FQ739bGvWbRvEW/d\n+RYd63e02SYz07hilphoVM6XuFEI55GamsqECRPo3r07kydPpl69ekRFRTFr1ix27NjB9u3bi3zu\n8OHDaWmVAJCcnEx4eDjDhg1j2LC8S7RBdryUtGrVKrTWduuvojjJJ6lzWZOVxZx9+/gjLQ1fV1fu\n9fdndnAwdzh6R1trWsOePXkZL3Fxxr4uzzxjpAxef72jRyisnEs6x7gvxnF3y7uZcrPtkqhaw+OP\nw65dRpbMdddV8iCFEMWqUaMGu3fvplu3brnHJk6cSLNmzXj55ZfZsWMH/fv3t/ncdu3a0a5du9zH\nCQkJhIeH06FDB0aNGlWq109LS6NmGTIuXZ3l86oEzj9n4wDbs7O51c+Pr9q3J65HD1a1bcvQwEDn\nCEJ+/92oatqiBdx6q7HL7dixRon0mBiYOVOCECdj1mbGbhyLm4sbHw35qMiCdW+9Zawdfu896NWr\nkgcphCiRu7t7viAkx9ChQ9Fa23UtRrdu3bj55pvZu3cvPXv2xMvLizlz5gCwbt06Bg0aRMOGDfH0\n9KRVq1bMnz+/0OzHyJEjCQkJyX38xx9/4OLiwpIlS1iyZAktWrSgZs2a3HrrrRw6dMhuYy8rmRGx\n4W1PT8LatHH0MPL8/bdRofHTT+HQIaOy6YgRRrpt796S8eLk3tj9Btv+2sa3Y74l0DvQZpvNm43y\nLdOnw7hx9h+DFPkUouKcO3cOgICAALv1qZTi/PnzDB48mIceeohx48bRqFEjAD788EPq1KnD9OnT\n8fLyYuvWrTz33HOkpqbmW4dScI1LjmXLlpGenk5ERAQmk4n58+czYsQIjh075pDK3hKI2ODmDHU0\nLl40anl/+qmR8eLpaaxgfPlluOsuyXi5Rvz8z888v+N5pt86nYEtBtps89tvMHKkUcJ93ryKGYcE\nGkJUnAULFuDn58ddd91l137Pnj3LihUrGDNmTL7j69evz7cw9tFHH2X8+PEsWrSIWbNmlbhA9fz5\n8xw7dgxvS1Xv4OBgRo4cyXfffVfkpaWKJIGIM0lNhS+/NIKPLVuMjJfbboMVK+C++4zdbsU1Iykj\nidB1oXSq34lX+79qs01cnBFfNm9uZFnL5JaoytLMZgBiUlPt3ndOnzmvUVnmzp3Ljh07WLp0KbXs\n/Dfa19eX0aNHFzpuHYQkJyeTkZFBz549WblyJcePH8+3KNaW0aNH5wYhAL169UJrzV9//SWBSLWU\nlZU/4yUlBW65xSge8cADIMV4rllT/juFc0nn2DxqMzVcC9eWycgwMqpTU2HnTvDxccAghahEJ9PT\nARhTgXUtTqan06OS0s1Wr17NzJkzefjhh5k0aZLd+2/SpInNSyWHDx/mxRdfZOfOnSQlJeUeV0px\n+fLlUvVrrU6dOgAkJpavRPvVkkDEEbSGqCgj+Fizxvha3KaNcdE+NNRYiCquaZG/RrLi0AqWD1lO\nS//C3060hkmT4Oef4fvv4RqslSREmQVbFiWtCgkhxMt2HZ3yiklNZUxMTO5rVLStW7cSFhbG4MGD\nWbp0aYW8hq0MmYSEBHr37k1QUBDz5s0jODgYT09PoqKieOmllzCXYkaoqGwaR6X6SiBSmY4cMebf\nIyPh5EmjYlVYmLHotFMn2eOlijiReILwr8MJbRfK2I5jbbZZsABWrjT+c7CxCF+IKqmmZe1CiJeX\n3aqfFvUaFWnv3r0MGzaMm2++mdWrV1dq0bBt27aRlJTE9u3b6Wq1r8uRI0cqbQz2JoFIRTt9Oi/j\n5fBhI+Pl/vth9GgjR/MaqHonSi/LlMWo9aOoW7MuS+9eanNadeNGeO45ePFFIwYVQlw7YmJiuOee\ne7juuuvYtGlTsdVUK0LObIb1zEdGRgbvvPNOpY7DniQQqQjx8bB2rRF8/PijkfFy770wZw7ccYdk\nvFRhs3fOZv/Z/fw04Sf8PAtfpz540IhBhw+HUlR7FkI4keTkZO644w4uXbrEM888w1dffZXvfIsW\nLWzWGbGn3r174+vrS2hoKFOmTCE7O5uVK1dWekBkTxKI2EtKSv6MF62NHctWrjQyXipoGlI4j+9P\nfs/cH+fyav9X6da48B+j8+eNDJmQECMRSibDhLi2JCQkcPbsWQBmzJhR6HxYWFiZApGi6nxYny+o\nXr16fPXVV0ybNo0XXniBunXrMn78eG655RYGDx5cYh9FvV5JY6lIEohcjawsY6/2Tz4x5ttTU6F7\nd3jzTePyi2S8VBsJqQmMWT+GPsF9eLbHs4XOp6XBkCFgNsMXX4Cd1+kJISpAWFgYYWFhuY+bNWuG\nyWSyS9/+/v7F9hUVFVXkuZ49e7Jnz55Cxwv2F5lTxdCidevWNl/Tw8PDbu+rPCQQKSuzGXbvzst4\nSUgwvuI+/7yR8SIbhFQ7WmsmfjmRtOw0Vg1dhauLa4HzMGEC/PqrcaXOUhxRCCEEThSIKKUigGlA\nfeAQMEVrvb+Ith8BYYAGrOeSjmit29toPxL4FNiotR5W8Hyp/PqrEXxERsKpU9C4sfHpMno0dOgg\nGS/V2Ds/v8MXf3zBxgc30qhW4Shjzhz47DOjUK7VInchqoXI2FgiL1wAIN1splXNmsz46y88Ldcm\nQ+vVI7Scs8cV2beoPE4RiCilHgTeACYB+4CpwDdKqVZa63gbT3kCsJ7/dgMOA2ts9B0MLAR+KPPA\nTp3Ky3j59VeoW9e45DJqFPTsKRf5Bb9d+I2nvn2KyTdOZkibIYXOr1kDs2YZwciIEQ4YoBAOFhoU\nVGHBQEX2LSqPUwQiGIHHu1rrlQBKqXDgbmACsKBgY611EpBbTk4pdR9QG1hu3U4p5QKsAl4CegOl\nKrdXe9s2ePddYz/2mjWNi/uvvWZkvNQoXCFTVE9pWWmErgulRZ0WvHH7G4XO79+fVybmhRccMEAh\nhLgGODwQUUq5A12BuTnHtNZaKbUN6F7KbiYA27TWfxc4PguI1Vp/pJTqXdox1V+1ygg6Pv7YyHiR\n2tvChulbp/Nnwp/sf2Q/Nd3zV0A8c8aIXzt1gmXL5MqdEEIUxeGBCBAAuAKxBY7HAq1LerJSqgFw\nFzCywPGewHigY1kH9L9Fi+gaEVHWp4lq5Ms/vuTt/W/zn7v+Q/ug/MuSUlKMsjFubkYyVSVVnBZC\niGuSMwQiV2sckAh8kXNAKeUDrAQe0VqXeRefbNnlVhTj7JWzTPhiAve2vpfHbnos3zmzGcaOhWPH\njOQquXwthBDFc4ZAJB4wAQX/ZAcB50vx/PHASq11ttWxFkAzYJPKq9DiAqCUygRaa61PFNXhG2++\nyerPP893LDQ0lNDQ0FIMR1RlJrOJsRvH4uHmwbJ7lxUqADRzprGJ8saNRjKVEEJURZGRkYXqlJRm\n519bHB6IaK2zlFIHgAHAlwCW4GEAsKi45yql+mIEHcsKnIoBCqbxvgb4YGTcFFxLks/TU6cy+qGH\nSvkORHWycPdCvjvxHdvGbiPAKyDfuVWrYO5cY0O7e+910ACFcICYmBhHD0FUsIL/xra+nEdHR+fb\niK+0HB6IWPwbWG4JSHLSd72wZMEopeYBDbXWYQWeNxHYq7XO9xvSWmcCv1sfU0pdMk5p+T9GlMu+\ns/uY+d1Mnu3xLP2b9893bvdumDgRxo+HadMcNEAhKllAQABeXl6MGTPG0UMRlcDLy4uAgICSG5aR\nUwQiWus1SqkA4BWMSzIHgTu01nGWJvWBJtbPUUrVAoZizHAIUaGuZFwhdF0oXRp04ZV+r+Q7d+qU\nkVx1yy2wdKlkyIjqo2nTpsTExBAfb6vck6hqAgICaNq0qd37dYpABEBrvQRYUsS58TaOXcG41FLa\n/gv1IURpRWyOIC4ljm/HfIu7q3vu8aQkuOceI8N7/XrZWFlUP02bNq2QDydRfThNICKEs1p1eBWr\nDq/i46Ef06Jui9zjJpNRrOz0aYiKggqYsRRCiCpPAhEhinH84nEmfz2ZMR3GMKZD/uvgzz4LmzfD\n119D27YOGqAQQlzjJBCxYXFGBqt//RWQTZOqsyxTFqPWj6Kedz3eHvR2vnPLlsEbb8Bbb8Gddzpo\ngEIIUQVIIGLDkpo16dK+0Ca+opp56buXiD4Xza4Ju6jlkVfkbudOCA+HRx+FKVMcOEAhhKgCyrV9\nrFKqn70HIoQz2f7Xdubvms+r/V7l5kY35x4/fhyGDYPevWHxYsmQEUKIq1Xefey3KKWOK6VeVEo1\nKbm5ENeOuJQ4HtrwEP2b92d6j+m5xy9dMjJk/P3h88/B3b2YToQQQpRKeQORRsB/gBHAX0qpb5RS\nDyilathvaEJUPq01E7+cSKYpk5VDV+KijP9FsrPhwQfh/Hn46iuoW9fBAxVCiCqiXIGI1jpea/2m\n1roTcAtwDKMGyD9KqUVKqTLveCuEM3h7/9tsOraJj4Z8REPfhrnHp06F7dth7Vpo1cqBAxRCiCqm\nvDMiubTW0cA8jBkSH2ACcEAp9aNS6oar7V+IynI49jDTvp3G4zc9zuDWg3OPL1kC//kPvP02DBjg\nwAEKIUQVVO5ARCnlrpQaoZTaDJwC7gAexyjRfr3l2OfFdCGE00jNSiV0XSit/Fux8PaFuce3boUn\nnjBujz7qwAEKIUQVVa70XaXUYiAUUMDHwDNa69+smqQopaYB/1z9EIWoeE9/8zR/Jf7FgUkH8HTz\nBODoUbj/fhg40KgZIoQQwv7KW0ekLTAFWK+1ziiiTTwgab7C6W2I2cA7B95h6d1LaRtolEhNSIDB\ng6FRI/jsM3CTijtCCFEhyvXnVWtd4pVyrXU2sLM8/QtRWc5cOcPDmx7mvjb38WhX49pLZiaMGAGJ\nibBvH/j5OXiQQghRhZW3oNlzSqlCu9kqpSYopZ69+mEJUfFMZhNj1o+hpltNPhj8AUoptIbHH4dd\nu2DDBrjuOkePUgghqrbyTjg/Cjxo4/gR4DNgfrlHJKq+yEjjBpCebuTDzpgBnsbaDEJDjVsF+7+f\n/o8fTv3AjrAd+Hv5A8beMe+/Dx99BL16VfgQhBCi2itvIFIfuGDjeBzQoPzDEdVCJQUaxYn6O4pZ\n38/i+V7P0ze4L2DspPv00zB9Oowb59DhCSFEtVHe9N2/gR42jvdAMmWEk7ucfplR60dxU6ObmNVn\nFgC//QYjRxol3OfNc/AAhRCiGinvjMj7wP9TSrkDOyzHBgALAEl0FE5La83krydzMe0iO8buwN3V\nnbg4I0OmeXP45BNwdXX0KIUQovoobyCyEPDHKOues79MOjBfay3fJ4XTWnloJZG/RfLpsE9pXqc5\nGRkwdCikpsLOneDj4+gRCiFE9VLe9F0NPKuUmgOEAGnAn8XUFBHC4f5M+JOIzRGEdQwjtH0oWsOk\nSfDzz/D999C0qaNHKIQQ1c9VlWnSWicD++00FiEqTKYpk9B1oTT0bcjiuxYDsGABrFxpXI7p1s3B\nAxRCiGqq3IGIUupG4AGgKXmXZwDQWg+7ynEJYVcv7niRw7GHiZoYha+HLxs3wnPPwYsvwqhRjh6d\nEEJUX+UtaDYS2I1xWWYo4A7cAPQHLtttdELYwbfHv2Xh7oXMHTCXrg27cvAgjB4Nw4fD7NmOHp0Q\nQlRv5Z0ReR6YqrV+WymVBDwJnADeBc7Za3CiaipYz+zUKWjWrGLqmV1IucDYDWO5vcXtPNX9Kc6f\nNzJkQkJgxQpwKff+00IIIeyhvIFIC+Bry8+ZgLfWWiul3sRI551lj8GJqsk60IiOhq5djcCkSxf7\nvo7WmvFfjMeszay4bwUZ6S4MGQJmM3zxBXh52ff1hBBClF15A5FEwNfy81mgHfArUBuQP+/CKSza\nu4jNf27m61FfE+Rdn1Gj4Ndf4ccfjV11hRBCOF55A5EfgIEYwcfnwFtKqf6WY9vtNDYhyu3g+YM8\ns6RIkpsAAB+SSURBVO0ZnrzlSQa1HMQrr8Bnn8HnnxszMEIIIZxDeQORxwHLFX1eA7KAW4F1wKt2\nGJcQ5ZaSmULoulBCAkKYf9t81qyBWbNgzhwYMcLRoxNCCGGtzEv1lFJuwD2ACUBrbdZa/5/W+l6t\n9dNa68TyDEQpFaGUOqGUSlNK7VFK3VRM24+UUmallMlyn3P71arNw0qpH5RSFy23rcX1KaqOqd9M\n5dSlU0QOj+TwLx6EhRkpui+84OiRCSGEKKjMgYjWOht4h7wZkaumlHoQY4+aWUBn4BDwjVIqoIin\nPIGxA3ADy31j4CKwxqpNH+BToC/QDWOjvm+VUrI7cBW29ve1vB/9Pm/d+Ra+GSEMGQKdOsGyZaCU\no0cnhBCioPImL+4DOtlxHFOBd7XWK7XWR4FwIBWYYKux1jpJa30h5wbcjLFQdrlVm4e01u9orQ9r\nrY8BD2O83wF2HLdwIqcvn+aRTY8wPGQ4oa0f5t57wc0NNm7MSw0WQgjhXMq7RmQJ8G+lVBPgAJBi\nfVJrfbi0HVl28O0KzLV6vlZKbQO6l7KbCcA2rfXfxbTxxii8drG0YxPXDpPZxJj1Y/Ct4cu7d79P\n2FjFsWOwaxcEBTl6dEIIIYpS3kDkM8v9IqtjGlCW+7JspB5gaR9b4Hgs0LqkJ1sutdwFjCyh6XyM\nVONtZRibuEa89uNr7Pp7F9+Hfc+/59ZhwwZjJqRjR0ePTAghRHHKG4g0t+sors44jLomXxTVQCk1\nA2NfnD5a68xKGpeoJLtO72L2ztm82OtFTv3Yi7lzjQ3t7r3X0SMTQghRknIFIlrrU3YcQzxGBk7B\nCfQg4Hwpnj8eWGlZRFuIUmoa8AwwQGt9pDQDmvrGG/itXp3vWGhoKKH2qjsu7OZS+iVGrR9F98bd\nGeA+k4ETYfx4mDbN0SMTQoiqKzIyksicvTosLl8u31Zz5QpElFJjizuvtV5Z2r601llKqQMYi0i/\ntPSvLI8XFfdcpVRfjHLzy4o4/wzwHHC71vqX0o7pzaefpsvo0aVtLhxEa82jXz3K5fTLfDrwB4b2\nc+OWW2DpUsmQEUKIimTry3l0dDRdy1ExsryXZt4q8Ngdo7R7Jka2S6kDEYt/A8stAck+jCwaLyxZ\nMEqpeUBDrXVYgedNBPZqrWMKdqiUehaYDYQCp5VSOTMuyVrrlILtxbXno4MfsebIGpbfvZrwkc3w\n8YH168HDw9EjE0IIUVrlvTRTp+AxpVRLYCmwsBz9rbHUDHkF45LMQeAOrXWcpUl9oEmB16sFDMWo\nKWJLOEaAtLbA8dmW1xHXsD/i/2DKf6cwvuME1s5+gNOnISoKAoqqPCOEEMIplXdGpBCt9f9v7+7j\nrCrrvY9/viBFiMJNJGgp3WSA3XrKITQ1rZOZJmWmlQ6Uih6U1DLU27Rz0o7PDykmRZnyIJkTkEel\nfMB8yocEkxHTNNRA1NRRBFFRFJnf+WOtgc12zzCzZ+9Ze/b+vl+vecFc+1prfnu/FH5c6/qu9WS6\nKfRqYEQRx08hiQUXem1cgbHXgL5tnK+SNtRaCb397tsceu2hbLvltmxx72VcdRPceCN84hNZV2Zm\nZh1V7A3NWvMusE2Jz2m2kdNuP43HXn6MQ3o2cNnFmzNpEuy3X9ZVmZlZMYrdrJofjBTJ7daPB+7r\nbFFmrbnlqVuYNH8Sx33sEs4dtzPHHAPf+17WVZmZWbGKvTRzfd73AbwM3AGc1KmKrOo1PNJAw6NJ\n7OulFWvg+GUce/8Qtlqc3Ie9fsd66nd6b1S66Y0mDr/+cPbaZj+uOeEE9toLJk92QsbMrDsrdrNq\nqS/pWA2p32lDo/HbOxpZsHwk39uhgbFfqGv1mOZo5vDrD4cQz/9iBgM/2IM5c6BXr66q2szMyqFk\nm1XNyunS+Zcy75/zqHvsFpYsG8SCBTBgQNZVmZlZZxW1siHpWkn/v8D4KZLmdL4ssw0aX2jk1NtO\n5VNvncjD1+7L738Pw4ZlXZWZmZVCsZdY9gJuKjB+c/qaWUm88c4b1F9bz+AeO7Lop+fyi1/A3ntn\nXZWZmZVKsZdm+pJEdfOtBbYsvhyzjZ1w8wksW/kca3/eyPePez/HHJN1RWZmVkrFrog8AhxSYPxQ\n4LHiyzHbYPbfZzNt0TR63DKZL9UN5+KLs67IzMxKrdgVkbOA/5H0MZLILiQPqasHvlmKwqy2Pf3q\n04yfezR9n/4W2746jt/dBJt5a7WZWdUpNr77B0kHAj8CvgG8BfwN+GJE/LmE9VkNerf5Xcb8fixv\nv9afPvMu54/3in79sq7KzMzKoeh/Y0bEjcCNJazFDIAz/3wW85+bT49Zd3PDrP4MHZp1RWZmVi7F\nxndHSdq1wPiukj7d+bKsVt2z7B7Ovvts4q4zuPL0Pdhzz6wrMjOzcip2s+ovKPxwuw+nr5l12Mq3\nVnLwNWOJZXtw0i7/yRFHZF2RmZmVW7GXZj4BLCow/lD6mlmHRASH/HY8L7/2Ol987WouuKJn1iWZ\nmVkXKHZF5G1gcIHxrSl8fxGzNt28dC5/+te1DFl0JdfN2I6e7kPMzGpCsY3IrcB5ktZnGST1B84F\n/lSKwqyGvNuLhmUX0fvvR3P35QfTt2/WBZmZWVcp9tLMycDdwDJJD6VjnwKagO+UojCrDW+vextW\nDybe6cvNJ0xiu+2yrsjMzLpSsfcR+ZekfwPGAp8kuY/IdKAhItaWsD6rcufPuxI2b+JbfX7F5/fo\nk3U5ZmbWxTpzH5HVku4FngHelw5/WRIRMbck1VlVO3n6bJ7cYho8sT8HHFVoy5GZmVW7ohoRSUOB\n64CdgACU/trCWw2tTVNvvZ+LlxzGB1/5Jq9sP4fkqQFmZlZrit2s+jNgKbAV8CawI/A54EHg8yWp\nzKrWPY8uYfwdB7DFa6M4d/RJxf9XaGZm3V6xfwXsBpweEcuBZmBdRNwLnAZcVqrirPosa1rJPtNH\n03NtfxaceB2bf6BX1iWZmVmGim1EegKvp79fzoa7rC4Dhne2KKtOq9e8Q9353+CdXi8x95Ab2WHI\nwKxLMjOzjBW7WfVRkrTMUmABcIqkd4CjgSUlqs2qSHNzMPKMCazY4h4m7XwbX95lWNYlmZlZBSi2\nETkb2Dz9/enAH4F7gFeAQ0pQl1WZr5x/Pov7TGf8B3/DD76+V9blmJlZhSj2PiLzcn7/FDBC0gBg\nZURE60daLTpp2ixuXvsjPrvuDH59/LezLsfMzCpIyfIKEbGiapqQyZPhgAOSr4aGrKvp1qbeej+X\nLD2cIavGctcZZ2RdjpmZVZiKCU5KOk7SUklvSZovaVQbc6dLapa0Lv215euRvHnflPR4es6HJX25\nXcVMmQJz5yZf9fWdfGe1Kzemu+i/p9Kzp7IuyczMKkxFNCKSDgEuBs4AdgYeBuZJai1W8X2Sp/9u\nnf76EWAFMDvnnLsD1wBXkDwH5wbgekmfKNPbsBzLmlbyxZyYbv8t3p91SWZmVoEqohEBJgKXR8TM\niPgHMIHkRmlHFpocEa9HxEstX8AuQH9gRs607wM3R8QlEbE4Ik4HGoHjy/lGbENMd61jumZmtgmZ\nNyKSegEjgdtbxtK9JreR3DitPY4EbouIZ3PGdkvPkWteB85pRWhuDurSmO4lu17nmK6ZmbWp6Ife\nldBAkhukNeWNN9GOm6NJ2hr4MnBo3kuDWzmnn65WRl85/3yecEzXzMzaqRIakc46AlhJsgekJCZO\nnEi/fv02Gquvr6feG1fb5JiumVltaGhooCEvVbpq1aqizlUJjchyYB0wKG98EPBiO44fB8yMiHfz\nxl8s9pyTJk2irq6uHT/aWkydl8Z0V4/lrosc0zUzq2aF/nHe2NjIyJEjO3yuzPeIRMRaYCGwd8uY\nJKXf/6WtYyV9HvgYMLXAy/fnnjO1TzpuJXTPo0sYf+fX2OJ1x3TNzKxjKmFFBOASYIakhcADJCma\nPqQpGEnnAdtExOF5xx0FLIiIxwuc82fAXZJOBG4E6kk2xY4vyzuoUetjuvTjAcd0zcysgyqiEYmI\n2ek9Q84kuXyyCNg3Il5OpwwGts09RtKWwNdJYrqFznm/pDHAOenXk8DXIuKx8ryL2rM+pvv+l7jx\noPsZsZ1jumZm1jEV0YgARMQUYEorr40rMPYa0HcT57wWuLYkBdpGcmO6HX2absMjDTQ8mmxyemnF\nGlg+jMmPn8qspt4A1O9YT/1O3hhsZlYLKqYRse6lMzHd+p02NBqNjTDyeJiyELw/2Mys9mS+WdW6\nnw0x3dMd0zUzs07xioh1SEtMd7vVY7jrop9kXY6ZmXVzbkSs3dbHdNeM4m9nT3NM18zMOs2NiLVL\nfky3X1/HdM3MrPPciNgmbRTTPdgxXTMzKx03Itam5uag7vScmO4oP03XzMxKx42ItWn0+efzxOZ+\nmq6ZmZWH47vWqpOmzeYWx3TNzKyMvCJiBSUx3cMc0zUzs7JyI2LvcfcjSxh/x9fY4m3HdM3MrLzc\niNhGljWtZJ8ZjumamVnXcCNi6zmma2ZmXc2NiAEtMd3vOqZrZmZdyo2IAS0x3WmO6ZqZWZdyfNcc\n0zUzs8x4RaTGOaZrZmZZciNSw7KK6TY0JF8Aa9bAsGFw6qnQu3cyVl+ffJmZWfVzI1KjsozputEw\nM7MWbkRqkGO6ZmZWKdyI1BjHdM3MrJK4EakxG2K6Mx3TNTOzzDm+W0M2jul+J+tyzMzMvCJSK668\nxTFdMzOrPG5EasDdjyzh6Dv9NF0zM6s8bkSq3NMv+mm6ZmZWudyIVLHVa96h7gLHdM3MrHK5EalS\nLTHdlY7pmplZBauY1Iyk4yQtlfSWpPmSRm1i/vsknSPpaUlrJC2RdETenB9I+oekNyU9I+kSSTVx\nbWL0eWlMd6upjumamVnFqogVEUmHABcDRwMPABOBeZKGRcTyVg6bA3wIGAf8E9ianMZK0hjgPOAI\n4H5gGDADaAZOLsf7qBQnTZvNLe86pmtmZpWvIhoRksbj8oiYCSBpAjAaOBK4MH+ypP2APYGhEfFq\nOvxM3rTdgHsjYlbL65J+B+xShvorRktMd4hjumZm1g1kfmlGUi9gJHB7y1hEBHAbSTNRyFeBB4Ef\nSnpO0mJJF0nqnTPnL8DIlks8koYC+wM3luFtVIT1Md3XR/HwmY7pmplZ5auEFZGBQE+gKW+8CRje\nyjFDSVZE1gAHpuf4JTAAOAogIhokDQTulaT0Z/wqIi4o+TuoAI7pmplZd1QJjUgxepDs9RgTEW8A\nSDoRmCPp2Ih4W9LngR8BE0j2nWwPXCbphYg4u62TT5w4kX79+m00Vl9fT32FPrveMV0zM+tKDQ0N\nNDQ0bDS2atWqos6l5CpIdtJLM28CB0fE3JzxGUC/iPh6gWNmALtHxLCcsRHA34FhEfFPSXcD8yPi\nlJw5Y0n2ovRtpZY6YOHChQupq6sryfsrt+bmYIdT/4Mnev+GS+tu44QDnZAxM7Ou19jYyMiRIwFG\nRkRje4/LfI9IRKwFFgJ7t4yll1L2JtnnUch9wDaS+uSMDSdZJXku/b4P8G7ecc05568K62O6g6a6\nCTEzs24n80YkdQkwXtJh6crGr0gaiRkAks6TdFXO/GuAV4DpknaQtBdJumZqRLydzvkDcKykQyR9\nVNI+wJnA3Mh6GahETpyaxHT3bD6dXx/nmK6ZmXU/FbFHJCJmpxtLzwQGAYuAfSPi5XTKYGDbnPmr\n08ZiMvBXkqZkFvDjnNOeRbICchbwYeBlYC7wX+V9N13jylvuZ9LTaUz3pz/JuhwzM7OiVEQjAhAR\nU4Aprbw2rsDYE8C+bZyvpQk5q1Q1Vorcp+k+fPY0evSomitNZmZWYyqmEbH22RDT3dIxXTMz6/bc\niHQj62O6vZu48aD5jumamVm350akm8h9mu6ldX6arpmZVQc3It3E+pjuwJmO6ZqZWdWolPiutcEx\nXTMzq1ZeEalwjumamVk1cyNSwRzTNTOzaudGpEI5pmtmZrXAjUgFyo3p3uSYrpmZVTE3IhUmP6a7\nn2O6ZmZWxdyIVJjR513gmK6ZmdUMx3crSBLTPc0xXTMzqxleEakQ62O6b47hrot+knU5ZmZmXcKN\nSAVoielu6ZiumZnVGDciGcuN6S5wTNfMzGqMG5EMOaZrZma1zo1IRpqbg7ofO6ZrZma1zY1IRkaf\ndwFP9J3G0Y7pmplZDXN8NwO5Md3LHdM1M7Ma5hWRLnaFY7pmZmbruRHpQn/+2xKOSWO6i86e6piu\nmZnVPDciXWRZ06t86aoNMd3+fXtnXZKZmVnm3Ih0gdVr3mHn8w92TNfMzCyPG5Eyc0zXzMysdW5E\nyswxXTMzs9Y5vltGjumamZm1zSsiZXLlLfMd0zUzM9uEilkRkXScpKWS3pI0X9KoTcx/n6RzJD0t\naY2kJZKOyJvTT9IvJD2fzvmHpP3K+kaAu/+2lKPvPIAtXx/Fov92TNfMzKw1FbEiIukQ4GLgaOAB\nYCIwT9KwiFjeymFzgA8B44B/AluT01hJ6gXcBrwIHAQ8DwwBXi3T2wCSmO4+V+1PT7bkgZMc0zUz\nM2tLRTQiJI3H5RExE0DSBGA0cCRwYf7kdFVjT2BoRLQ0Fs/kTTsK6A98JiLWtTKnpPJjusO3dUzX\nzMysLZlfmklXLkYCt7eMRUSQrGbs1sphXwUeBH4o6TlJiyVdJKl33pz7gSmSXpT0iKTTJJXlPW8U\n0/3M9Y7pmpmZtUMlrIgMBHoCTXnjTcDwVo4ZSrIisgY4MD3HL4EBJCshLXO+AFwNfBnYPp2zGXBW\n6cpP5MZ0v/81x3TNzMzaoxIakWL0AJqBMRHxBoCkE4E5ko6NiLfTOU3A0ekKy0OSPgKcTIkbkYlX\nOqZrZmZWjEpoRJYD64BBeeODSDaaFvIC8K+WJiT1OCDgIySbV18A3kmbkNw5gyVtFhHvtlbQxIkT\n6dev30Zj9fX11NfXv2fulbfM59JljumamVntaGhooKGhYaOxVatWFXWuzBuRiFgraSGwNzAXQJLS\n7y9r5bD7gG9I6hMRb6Zjw0lWSZ7LmZPfOQwHXmirCQGYNGkSdXV1m6x9fUzXT9M1M7MaUugf542N\njYwcObLD58p8s2rqEmC8pMMkjQB+BfQBZgBIOk/SVTnzrwFeAaZL2kHSXiTpmqnpZRlI94xIukzS\nxyWNBk4Dfl6KgtfHdN91TNfMzKxYma+IAETEbEkDgTNJLsksAvaNiJfTKYOBbXPmr5a0DzAZ+CtJ\nUzIL+HHOnOck7QtMAh4G/pX+/j1x4I5yTNfMzKw0KqIRAYiIKcCUVl4bV2DsCWDfTZxzAbB7SQpM\n5cZ0fzbST9M1MzPrjIppRLqL/c9NYrrjB17lmK6ZmVknVcoekW7hxKlzmLcuien++rjDsi7HzMys\n2/OKSDslT9P9jmO6ZmZmJeRGpB0c0zUzMysPNyKb4KfpmpmZlY8bkTbkxnRvPtgxXTMzs1JzI9KK\n/Jjuvp92TNfMzKzU3Ii0wjFdMzOz8nN8t4CLr/8T89adxl7hmK6ZmVk5eUWkgGue/zFD+o3hTsd0\nzczMysorIgX0WPH/GLF4KgceKPKecmxmZmYl5BWRAj72f3vyvk9/K/lmx3qgvs35ZmZmVhw3IgUc\n9vGJ/Ff92KzLMDMzq3puRAqY9fRkHmiYBUD9jvXU7+QVETMzs3JwI1LA5ndNgcfqkm/qgZ0yLcfM\nzKxquREpYMoUqKvLugozM7Pq59SMmZmZZcaNiJmZmWXGjYiZmZllxo2ImZmZZcaNiJmZmWXGjYiZ\nmZllxo2ImZmZZcaNiJmZmWXGjYiZmZllxo2ImZmZZcaNiJmZmWXGjYiZmZllpmIaEUnHSVoq6S1J\n8yWN2sT890k6R9LTktZIWiLpiFbmHiqpWdL/lKV465SGhoasS6g5/sy7nj/zrufPvHuoiEZE0iHA\nxcAZwM7Aw8A8SQPbOGwO8O/AOGAYUA8sLnDujwIXAXeXtGgrGf9h0fX8mXc9f+Zdz59597BZ1gWk\nJgKXR8RMAEkTgNHAkcCF+ZMl7QfsCQyNiFfT4WcKzOsBXA2cDuwF9CtL9WZmZlaUzFdEJPUCRgK3\nt4xFRAC3Abu1cthXgQeBH0p6TtJiSRdJ6p037wygKSKml6F0MzMz66RKWBEZCPQEmvLGm4DhrRwz\nlGRFZA1wYHqOXwIDgKMAJH2W5LLNJ0tfspmZmZVCJTQixegBNANjIuINAEknAnMkHQv0AmYC4yNi\nZQfO2xvg8ccfL3G51pZVq1bR2NiYdRk1xZ951/Nn3vX8mXetnL87869OtEnJVZDspJdm3gQOjoi5\nOeMzgH4R8fUCx8wAdo+IYTljI4C/k2xc7Qs0AusApVNaLkOtA4ZHxNIC5x0D/Lbz78rMzKxmjY2I\na9o7OfMVkYhYK2khsDcwF0CS0u8va+Ww+4BvSOoTEW+mY8NJVkmeS7/fKe+Yc0galO8Dz7Zy3nnA\nWOBpkss+ZmZm1j69gY+S/F3abpmviABI+hYwA5gAPECSovkGMCIiXpZ0HrBNRByezt8ceAyYD/wE\n+BBwBXBnRExo5WdMJ1lhOai878bMzMzaK/MVEYCImJ3eM+RMYBCwCNg3Il5OpwwGts2Zv1rSPsBk\n4K/AK8As4MddWriZmZl1SkWsiJiZmVltyvw+ImZmZla73IiYmZlZZtyItELSDZKWpQ/he17STElb\nZ11XtZI0RNKV6cML35T0pKSfpPFuKxNJP5J0n6TVklZkXU816ugDPa1zJO0paa6kf6UPOz0g65qq\nmaTTJD0g6TVJTZKukzRs00du4EakdXcA3yS5L8lBwMdIHrRn5TGC5J4v44FPkCSnJpDErq18egGz\nSe5MbCVW5AM9rXM2Jwk8HAt4E2T57UkSHNkV+CLJnym3SvpAe0/gzartJOmrwHXA+yNiXdb11AJJ\nJwMTImL7rGupdpIOByZFxICsa6kmkuYDCyLihPR7kdzH6LKIeM8DPa20JDUDB+beLNPKK22yXwL2\nioh723OMV0TaQdIAkhud3ecmpEv1B3y5wLqlIh/oadbd9SdZiWr3n91uRNog6XxJbwDLSe5jcmDG\nJdUMSdsDxwO/yroWsyK19UDPwV1fjll5pSt+lwL3RsRj7T2uphoRSeelm5da+1qXt8nmQuBTwD4k\nz6j5TSaFd2NFfOZI+jBwMzArIqZlU3n3VcxnbmZWAlNI9vgd2pGDKuLOql3op8D0TcxZ0vKbiFhB\nsrz0lKR/AM9K2jUiFpSxxmrToc9c0jYkG4XvjYhjyllYFevQZ25ls5zkHzCD8sYHAS92fTlm5SPp\n58D+wJ4R8UJHjq2pRiQiXiG5HXwxeqa/vr9E5dSEjnzm6UrIHSS37T+ynHVVs07+d24lUuQDPc26\nnbQJ+RrwuYh4pqPH11Qj0l6SdgFGAfcCK4HtSZ6D8yRwf4alVa10JeQuYClwCrBV8mc2RET+NXYr\nEUnbAgOAIUBPSZ9MX3oqIlZnV1nVuASYkTYkLQ/07EPykE8rg/ShqNuT3A4AYGj63/WKiGjtyetW\nJElTgHrgAGC1pJYVwFUR0a6n2Du+W4CkHYGfAf9Gkkl/gWTPwjkdXXKy9knjo/n7QUQSNOhZ4BAr\ngfSp1IcVeOnfI+Lurq6nGkk6lqS5bnmg5/ci4sFsq6pekj4H3Ml77yFyVUR4pbXE0oh0oUZiXETM\nbNc53IiYmZlZVmoqNWNmZmaVxY2ImZmZZcaNiJmZmWXGjYiZmZllxo2ImZmZZcaNiJmZmWXGjYiZ\nmZllxo2ImZmZZcaNiJmZmWXGjYiZlY2kOyVdUobz/llShx41LmkHSc9K+kCp6zGz4rkRMbNuRdIB\nwFYR8bu88Z0lzZb0oqS3JC2W9GtJHweIiMdJHlp5UgZlm1kr3IiYWXfzPWB67oCkr5A0Gb2AMcAI\n4NvAqyRPzm4xA/iuJP/ZZ1Yh/D+jmXUJSf0lzZS0QtJqSTdJ2j5vznhJz0h6I13d+IGklTmvDwS+\nAPwhZ+wDJE9u/mNEfD0i7oiIZRHx14g4BTgm50f8CRgAfK6c79XM2s+NiJl1lauAOuArwGcAATdJ\n6gkgaQ/gl8Ak4FPAHcB/svEjxj8LrE4vs7TYD/ggcGGhHxoRr+X8fi2wCNizNG/JzDprs6wLMLPq\nl658fBXYLSIWpGNjgWeBA4FrgeOBmyJiUnrYU2lzMjrnVEOAprzTt6yqLG5nOc+n5zGzCuAVETPr\nCjsAa4EHWgYiYgVJ87BDOjQ89/VU/vcfANbkjamDtbwF9OngMWZWJm5EzKw7WQ78n7yxJ9JfR7Tz\nHAOAl0tWkZl1ihsRM+sKj5MkWnZtGZD0QZJVkL+nQ4uBUXnH7ZL3/UPAYEn9csZuBV4BTin0g/Pm\nAuyYnsfMKoAbETMru4h4CrgBuELSHpI+CVxNskdkbjptMrC/pImStpd0DMlG1NzNqg+RrIrskXPu\nN4H/AEZLukHS3pKGSBop6QKSDbAASBoCbAPcVrY3a2Yd4kbEzMopt4kYBywkid7eBzQDoyNiHUBE\n/AWYAEwkSbZ8iSRBs35PSEQ0k9wL5Nsb/ZCIucDuwDvAb0lWYK4BBrHxfUTGALdGxLOleoNm1jmK\niE3PMjPLgKQrgGER8bmcsUHAo0BdRxoKSb2AJ4FDI2J+yYs1s6I4vmtmFUPSSSQ3HVsN7A98B/hu\n7pyIaJJ0FLAdyaWd9toOOMdNiFll8YqImVUMSbNI7nq6BbAEuCwirsi2KjMrJzciZmZmlhlvVjUz\nM7PMuBExMzOzzLgRMTMzs8y4ETEzM7PMuBExMzOzzLgRMTMzs8y4ETEzM7PMuBExMzOzzPwvgme8\nPVjfbaUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x27db2c06320>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot CV误差曲线\n",
    "test_means = grid.cv_results_[ 'mean_test_score' ]\n",
    "train_means = grid.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = grid.cv_results_[ 'std_train_score' ]\n",
    "test_stds = grid.cv_results_[ 'std_test_score' ]\n",
    "\n",
    "\n",
    "# plot results\n",
    "n_Cs = len(Cs)\n",
    "number_penaltys = len(penaltys)\n",
    "test_scores = np.array(test_means).reshape(n_Cs,number_penaltys)\n",
    "train_scores = np.array(train_means).reshape(n_Cs,number_penaltys)\n",
    "test_stds = np.array(test_stds).reshape(n_Cs,number_penaltys)\n",
    "train_stds = np.array(train_stds).reshape(n_Cs,number_penaltys)\n",
    "\n",
    "\n",
    "\n",
    "x_axis = np.log10(Cs)\n",
    "for i, value in enumerate(penaltys):\n",
    "    #pyplot.plot(log(Cs), test_scores[i], label= 'penalty:'   + str(value))\n",
    "    plt.errorbar(x_axis, test_scores[:,i], yerr=test_stds[:,i] ,label = penaltys[i] +' Test')\n",
    "    plt.errorbar(x_axis, train_scores[:,i], yerr=train_stds[:,i] ,label = penaltys[i] +' Train')\n",
    "    \n",
    "plt.legend()\n",
    "plt.xlabel( 'log(C)' )                                                                                                      \n",
    "plt.ylabel( 'accuracy' )\n",
    "plt.savefig('LogisticGridSearchCV_C.png' )\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上图给出了L1正则和L2正则下、不同正则参数C对应的模型在训练集上和测试集上的accuracy\n",
    "C较小时L2正则比L1正则好很多，C逐渐增大，L2正则和L1正则基本一致\n",
    "在测试集上当C=1时性能最好（L1正则）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 保存模型，用于后续测试"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pickle\n",
    "\n",
    "pickle.dump(grid.best_estimator_, open(\"diabetes_L1_org_accuracy_loss.pkl\", 'wb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:Anaconda3]",
   "language": "python",
   "name": "conda-env-Anaconda3-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
